[ISSUEE 60] Add features to the broker and nameserver - #61
Conversation
linjiemiao
commented
Sep 25, 2020
- add affinity / securityContext / imagePullSecrets / tolerations / nodeSelector / podAnnotations / priorityClassName
nodeSelector / podAnnotations / priorityClassName
| Affinity: broker.Spec.Affinity, | ||
| SecurityContext: broker.Spec.SecurityContext, | ||
| ImagePullSecrets: broker.Spec.ImagePullSecrets, | ||
| Tolerations: broker.Spec.Tolerations, | ||
| NodeSelector: broker.Spec.NodeSelector, | ||
| PriorityClassName: broker.Spec.PriorityClassName, |
There was a problem hiding this comment.
You need to write functions for each of them and handle nil value scenarios, by directly adding this you are forcing the user at all times to specify these values, the operator will crash if you don't pass in tolerations.
@liuruiyiyang since these values will go to both broker/nameserver we can create single functions which can be used both in broker and nameserver
There was a problem hiding this comment.
+1 This pr added features that help enrich operator, hope that corresponding null check would be supplemented.
| Affinity: nameService.Spec.Affinity, | ||
| SecurityContext: nameService.Spec.SecurityContext, | ||
| ImagePullSecrets: nameService.Spec.ImagePullSecrets, | ||
| Tolerations: nameService.Spec.Tolerations, | ||
| NodeSelector: nameService.Spec.NodeSelector, | ||
| PriorityClassName: nameService.Spec.PriorityClassName, |
There was a problem hiding this comment.
same here, we need to write functions to get values to handle nil/empty scenarios
| Affinity: broker.Spec.Affinity, | ||
| SecurityContext: broker.Spec.SecurityContext, | ||
| ImagePullSecrets: broker.Spec.ImagePullSecrets, | ||
| Tolerations: broker.Spec.Tolerations, | ||
| NodeSelector: broker.Spec.NodeSelector, | ||
| PriorityClassName: broker.Spec.PriorityClassName, |
There was a problem hiding this comment.
+1 This pr added features that help enrich operator, hope that corresponding null check would be supplemented.
| NodeSelector map[string]string `json:"nodeSelector,omitempty"` | ||
| // PodAnnotations you can use annotations to attach arbitrary non-identifying metadata to objects. | ||
| PodAnnotations map[string]string `json:"podAnnotations,omitempty"` | ||
| // PriorityClassName defines priority class's name | ||
| PriorityClassName string `json:"priorityClassName,omitempty"` |
There was a problem hiding this comment.
It seems null checks are required for them, would define them as pointer be better?
| NodeSelector map[string]string `json:"nodeSelector,omitempty"` | ||
| // PodAnnotations you can use annotations to attach arbitrary non-identifying metadata to objects. | ||
| PodAnnotations map[string]string `json:"podAnnotations,omitempty"` | ||
| // PriorityClassName defines priority class's name | ||
| PriorityClassName string `json:"priorityClassName,omitempty"` |
There was a problem hiding this comment.
It seems null checks are required for them, would define them as pointer be better?
|
This PR has conflicts with the base branch and cannot be merged. Please rebase or merge the base branch into your branch and resolve the conflicts: git fetch origin
git checkout issue-60-dev
git rebase origin/main
# resolve conflicts, then:
git push --force-with-leaseThis is a one-time reminder. Feel free to @mention me for a re-review after conflicts are resolved. Automated notification by github-manager-bot |
RockteMQ-AI
left a comment
There was a problem hiding this comment.
Summary
This PR modifies 7 file(s) with 351 lines of diff. No test changes detected — consider adding test coverage.
Automated review by github-manager-bot
| @@ -56,6 +56,21 @@ type BrokerSpec struct { | |||
| VolumeClaimTemplates []corev1.PersistentVolumeClaim `json:"volumeClaimTemplates"` | |||
There was a problem hiding this comment.
No test changes detected alongside source modifications. Consider adding tests to cover the changes.
|
This PR has been open since September 2020 (nearly 6 years) and currently has merge conflicts with the master branch. Status check: Is this PR still actively being worked on? The feature additions (affinity, tolerations, nodeSelector, etc.) are valuable, but the code needs to be rebased to resolve conflicts. If this is still relevant, please:
If this has been abandoned, consider closing it. Automated review by github-manager-bot |
RockteMQ-AI
left a comment
There was a problem hiding this comment.
Summary
PR received and logged for review. This PR requires detailed code review by a maintainer.
Diff size: 351 lines
Author: linjiemiao (NONE)
Automated review by RockteMQ-AI
RockteMQ-AI
left a comment
There was a problem hiding this comment.
Summary
Review of PR #61: [ISSUEE 60] Add features to the broker and nameserver
Findings: 4 issue(s) identified (1 critical).
CLA: unknown
Please address the inline comments above.
Automated review by github-manager-bot
| VolumeClaimTemplates []corev1.PersistentVolumeClaim `json:"volumeClaimTemplates"` | ||
| // The name of pod where the metadata from | ||
| ScalePodName string `json:"scalePodName"` | ||
| // Affinity, affinity and anti-affinity scheduling |
There was a problem hiding this comment.
New pointer/slice/map fields (Affinity, SecurityContext, ImagePullSecrets, Tolerations, NodeSelector, PodAnnotations) were added to BrokerSpec, but the PR does not include updates to zz_generated_deepcopy.go. If the generated DeepCopyInto is not regenerated, these fields are shallow-copied: Affinity/SecurityContext pointers and NodeSelector/PodAnnotations maps will be shared between the informer cache and the object handed to the reconciler. Any mutation corrupts the shared cache and can cause cross-reconcile state corruption. Run the operator-sdk/k8s codegen to regenerate deepcopy for both broker_types.go and nameservice_types.go.
| Template: corev1.PodTemplateSpec{ | ||
| ObjectMeta: metav1.ObjectMeta{ | ||
| Labels: ls, | ||
| Annotations: broker.Spec.PodAnnotations, |
There was a problem hiding this comment.
The new pod-scheduling fields are wired into getBrokerStatefulSet (Annotations, Affinity, SecurityContext, ImagePullSecrets, Tolerations, NodeSelector, PriorityClassName) and similarly into nameservice_controller.go, but no test changes accompany this PR. There is no coverage verifying that CR-spec values propagate to the rendered StatefulSet PodTemplateSpec, nor that nil/empty values (e.g. unset Affinity, empty PriorityClassName) render safely. Add reconciler tests asserting these fields pass through correctly.
| prometheus.io/scrape: "true" | ||
| securityContext: | ||
| allowPrivilegeEscalation: true | ||
| runAsUser: 0 |
There was a problem hiding this comment.
The NameService securityContext sets allowPrivilegeEscalation: true, runAsUser: 0, runAsGroup: 0. This example is intended as a copyable template, and running the name server as root with privilege escalation enabled is a poor security pattern for an Apache-shipped reference. Consider a non-root runAsUser (the alpine image supports a dedicated UID) and allowPrivilegeEscalation: false, or at minimum document the risk and make it opt-in.
| items: | ||
| type: object | ||
| type: array | ||
| affinity: |
There was a problem hiding this comment.
The new complex Kubernetes types (affinity, securityContext, tolerations, imagePullSecrets items) are declared as bare type: object/item type: object with no OpenAPI v3 properties. This provides no server-side validation, so malformed scheduling rules are accepted silently and only surface as Pod creation errors later. For a cleaner CRD (and better kubectl explain output), define the full schema via x-kubernetes-preserve-unknown-fields or the concrete property trees. Same applies to rocketmq_v1alpha1_nameservice_crd.yaml.
RockteMQ-AI
left a comment
There was a problem hiding this comment.
Summary
The PR correctly adds standard Kubernetes scheduling fields to both CRDs and controllers with consistent Go types, but lacks CRD schema validation, has a security-concerning example, and misses test coverage and annotation merge logic.
Findings
- [WARNING]
deploy/crds/rocketmq_v1alpha1_broker_crd.yaml:84— Theaffinityfield is declared astype: objectwith no structural schema. This allows any arbitrary YAML to pass CRD validation, deferring errors to the kube-scheduler at runtime. Consider usingx-kubernetes-preserve-unknown-fields: truewith a more specific schema, or at minimum document the expected structure. Same issue applies tosecurityContext,nodeSelector, andpodAnnotations(alltype: object), and toimagePullSecrets/tolerations(items: type: object). While this pattern is common in early-stage operators, it provides zero admission-time validation. - [WARNING]
deploy/crds/rocketmq_v1alpha1_nameservice_crd.yaml:68— Same unstructured schema issue as the broker CRD —affinity,securityContext,nodeSelector,podAnnotationsall accept arbitrary objects. Since these fields are duplicated across two CRDs, consider extracting a shared validation schema (or a shared struct with code-generated OpenAPI) to keep them in sync and avoid drift. - [WARNING]
example/rocketmq_v1alpha1_rocketmq_feature_cluster.yaml:128— The example setsallowPrivilegeEscalation: trueandrunAsUser: 0/runAsGroup: 0, which runs the NameService container as root with privilege escalation enabled. This is a security anti-pattern and should not be in an example file that users will copy. If root is truly required for RocketMQ, add a comment explaining why and recommend dropping to a non-root user where possible. At minimum, setallowPrivilegeEscalation: false. - [WARNING]
pkg/controller/broker/broker_controller.go:404—Annotations: broker.Spec.PodAnnotationsdirectly assigns the spec map to the pod template. If a user later removes all annotations from the spec (setting it tonilor{}), the existing annotations on the live StatefulSet's pod template won't be cleared during an update unless the controller explicitly handles this in its update/reconciliation path. Verify that the update logic inupdateBrokerStatefulSet(or equivalent) properly propagates annotation removal, not just addition. - [INFO]
pkg/controller/broker/broker_controller.go:405— All six new PodSpec fields (Affinity,SecurityContext,ImagePullSecrets,Tolerations,NodeSelector,PriorityClassName) are assigned directly from the CR spec with no nil-guard. This is safe for the Kubernetes API (nil/empty values are valid), but it means any change to these fields will trigger a StatefulSet rolling update. For fields likenodeSelectorortolerationsthat an operator might want to change frequently, consider documenting this behavior so users understand the pod churn implications. - [WARNING]
pkg/controller/nameservice/nameservice_controller.go:319— Same annotation propagation concern as the broker controller:Annotations: nameService.Spec.PodAnnotationsis a direct assignment. If the existing StatefulSet's pod template has annotations set by a mutating webhook or by a previous spec version, they will be silently overwritten on reconciliation. Consider merging annotations (existing + spec) rather than replacing, or document that spec annotations are authoritative. - [INFO]
pkg/apis/rocketmq/v1alpha1/broker_types.go:72— No test coverage is added for the new fields. The broker and nameservice controller tests should verify that: (1) a CR with these fields produces a StatefulSet with the correct PodSpec, (2) nil/empty values produce the same StatefulSet as before (backward compat), and (3) updates to these fields trigger a rolling update. Without tests, regressions in the reconciliation logic are likely to go undetected.
Automated review by github-manager-bot
| type: array | ||
| affinity: | ||
| description: Affinity and anti-affinity scheduling | ||
| type: object |
There was a problem hiding this comment.
The affinity field is declared as type: object with no structural schema. This allows any arbitrary YAML to pass CRD validation, deferring errors to the kube-scheduler at runtime. Consider using x-kubernetes-preserve-unknown-fields: true with a more specific schema, or at minimum document the expected structure. Same issue applies to securityContext, nodeSelector, and podAnnotations (all type: object), and to imagePullSecrets/tolerations (items: type: object). While this pattern is common in early-stage operators, it provides zero admission-time validation.
| type: array | ||
| affinity: | ||
| description: Affinity and anti-affinity scheduling | ||
| type: object |
There was a problem hiding this comment.
Same unstructured schema issue as the broker CRD — affinity, securityContext, nodeSelector, podAnnotations all accept arbitrary objects. Since these fields are duplicated across two CRDs, consider extracting a shared validation schema (or a shared struct with code-generated OpenAPI) to keep them in sync and avoid drift.
| - azure-k8s-2 | ||
| podAntiAffinity: | ||
| preferredDuringSchedulingIgnoredDuringExecution: | ||
| - weight: 100 |
There was a problem hiding this comment.
The example sets allowPrivilegeEscalation: true and runAsUser: 0 / runAsGroup: 0, which runs the NameService container as root with privilege escalation enabled. This is a security anti-pattern and should not be in an example file that users will copy. If root is truly required for RocketMQ, add a comment explaining why and recommend dropping to a non-root user where possible. At minimum, set allowPrivilegeEscalation: false.
| Template: corev1.PodTemplateSpec{ | ||
| ObjectMeta: metav1.ObjectMeta{ | ||
| Labels: ls, | ||
| Annotations: broker.Spec.PodAnnotations, |
There was a problem hiding this comment.
Annotations: broker.Spec.PodAnnotations directly assigns the spec map to the pod template. If a user later removes all annotations from the spec (setting it to nil or {}), the existing annotations on the live StatefulSet's pod template won't be cleared during an update unless the controller explicitly handles this in its update/reconciliation path. Verify that the update logic in updateBrokerStatefulSet (or equivalent) properly propagates annotation removal, not just addition.
| ObjectMeta: metav1.ObjectMeta{ | ||
| Labels: ls, | ||
| Annotations: broker.Spec.PodAnnotations, | ||
| }, |
There was a problem hiding this comment.
All six new PodSpec fields (Affinity, SecurityContext, ImagePullSecrets, Tolerations, NodeSelector, PriorityClassName) are assigned directly from the CR spec with no nil-guard. This is safe for the Kubernetes API (nil/empty values are valid), but it means any change to these fields will trigger a StatefulSet rolling update. For fields like nodeSelector or tolerations that an operator might want to change frequently, consider documenting this behavior so users understand the pod churn implications.
| Template: corev1.PodTemplateSpec{ | ||
| ObjectMeta: metav1.ObjectMeta{ | ||
| Labels: ls, | ||
| Annotations: nameService.Spec.PodAnnotations, |
There was a problem hiding this comment.
Same annotation propagation concern as the broker controller: Annotations: nameService.Spec.PodAnnotations is a direct assignment. If the existing StatefulSet's pod template has annotations set by a mutating webhook or by a previous spec version, they will be silently overwritten on reconciliation. Consider merging annotations (existing + spec) rather than replacing, or document that spec annotations are authoritative.
| // PodAnnotations you can use annotations to attach arbitrary non-identifying metadata to objects. | ||
| PodAnnotations map[string]string `json:"podAnnotations,omitempty"` | ||
| // PriorityClassName defines priority class's name | ||
| PriorityClassName string `json:"priorityClassName,omitempty"` |
There was a problem hiding this comment.
No test coverage is added for the new fields. The broker and nameservice controller tests should verify that: (1) a CR with these fields produces a StatefulSet with the correct PodSpec, (2) nil/empty values produce the same StatefulSet as before (backward compat), and (3) updates to these fields trigger a rolling update. Without tests, regressions in the reconciliation logic are likely to go undetected.